ISRN  TELCORDIA— 2013-1 1+PR-OGARAU 
Prepared  for:  Office  of  Naval  Research 


Fast  Multiscale  Algorithms  for 
Information  Representation  and  Fusion 

Technical  Progress  Report  No.  1 1 

Devasis  Bassu,  Principal  Investigator 


Contract:  N00014-10-C-0176 
Applied  Communication  Sciences 
150  Mount  Airy  Road 
Basking  Ridge,  NJ  07920-2021 

April  2013 


Approved  for  public  release;  distribution  is  unlimited. 


Report  Documentation  Page 

Form  Approved 

OMB  No.  0704-0188 

Public  reporting  burden  for  the  collection  of  information  is  estimated  to  average  1  hour  per  response,  including  the  time  for  reviewing  instructions,  searching  existing  data  sources,  gathering  and 
maintaining  the  data  needed,  and  completing  and  reviewing  the  collection  of  information.  Send  comments  regarding  this  burden  estimate  or  any  other  aspect  of  this  collection  of  information, 
including  suggestions  for  reducing  this  burden,  to  Washington  Headquarters  Services,  Directorate  for  Information  Operations  and  Reports,  1215  Jefferson  Davis  Highway,  Suite  1204,  Arlington 

VA  22202-4302.  Respondents  should  be  aware  that  notwithstanding  any  other  provision  of  law,  no  person  shall  be  subject  to  a  penalty  for  failing  to  comply  with  a  collection  of  information  if  it 
does  not  display  a  currently  valid  OMB  control  number. 

1.  REPORT  DATE 

APR  2013  2.  REPORT  TYPE 

3.  DATES  COVERED 

00-00-2013  to  00-00-2013 

4.  TITLE  AND  SUBTITLE 

Fast  Multiscale  Algorithms  for  Information  Representation  and  Fusion 

5a.  CONTRACT  NUMBER 

5b.  GRANT  NUMBER 

5c.  PROGRAM  ELEMENT  NUMBER 

6.  AUTHOR(S) 

5d.  PROJECT  NUMBER 

5e.  TASK  NUMBER 

5f.  WORK  UNIT  NUMBER 

7.  PERFORMING  ORGANIZATION  NAME(S)  AND  ADDRESS(ES) 

Applied  Communication  Sciences, 150  Mount  Airy  Road, Basking 

Ridge, NJ, 07920 

8.  PERFORMING  ORGANIZATION 

REPORT  NUMBER 

9.  SPONSORING/MONITORING  AGENCY  NAME(S)  AND  ADDRESS(ES) 

10.  SPONSOR/MONITOR'S  ACRONYM(S) 

11.  SPONSOR/MONITOR'S  REPORT 
NUMBER(S) 

12.  DISTRIBUTION/AVAILABILITY  STATEMENT 

Approved  for  public  release;  distribution  unlimited 

13.  SUPPLEMENTARY  NOTES 

14.  ABSTRACT 

15.  SUBJECT  TERMS 

16.  SECURITY  CLASSIFICATION  OF:  17.  LIMITATION  OF 

_ _ _  ABSTRACT 

18.  NUMBER  19a.  NAME  OF 

OF  PAGES  RESPONSIBLE  PERSON 

a.  REPORT  b.  ABSTRACT  c.  THIS  PAGE  Same  US 

unclassified  unclassified  unclassified  Report  (SAR) 

12 

Standard  Form  298  (Rev.  8-98) 

Prescribed  by  ANSI  Std  Z39-18 


ArrUtU 

Communication 

sciences 


ISRN  TELCORDIA-2013-1 1+PR-OGARAU 
Technical  Progress  Report 
Abstract 


1  Abstract 


In  the  eleventh  quarter  of  the  work  effort,  we  focused  on  a)  conducting  experiments  on  real- 
world  data  sets  using  the  developed  algorithms,  b)  continued  design/implementation  of  the 
Multiscale  Heat-Kernel  Coordinates  (MHKC)  algorithms  with  function  estimation  and  c) 
packaging  for  releasing  the  software  as  open  source.  This  report  documents  algorithm  designs  for 
the  MHKC  algorithms  for  visualization  and  function  estimation. 

The  project  is  currently  on  track  -  in  the  upcoming  final  quarter,  we  will  wrap  up  all  pending 
work,  write  the  final  report  and  package  all  the  deliverables.  No  problems  are  currently 
anticipated. 


Use  or  disclosure  of  data  contained  on  this  sheet  is  subject  to  restrictions  on  the  title  page  of  this  report. 
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2  Summary 


In  this  quarter,  we  continued  design  and  implementation  of  the  new  multiscale  heat  kernel 
coordinates  (MHKC)  algorithms  for  visualization  and  function  extension.  The  design  variants  for 
MHKC  algorithms  are  documented  in  this  report. 

The  project  is  currently  on  track  -  in  the  upcoming  final  quarter,  we  will  wrap  up  all  pending 
work,  write  the  final  report  and  package  all  the  deliverables.  No  problems  are  currently 
anticipated. 


Use  or  disclosure  of  data  contained  on  this  sheet  is  subject  to  restrictions  on  the  title  page  of  this  report. 
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3  Introduction 


The  primary  project  effort  over  the  last  quarter  focused  on  completing  design/development  of  the 
multiscale  heat-kernel  coordinates  algorithms  [1],  An  outline  of  the  MHKC  algorithms  was 
presented  in  previous  quarterly  reports  [  10]  [  1 1] .  This  reports  presents  the  MHKC  algorithms  for 
obtaining  stable  coordinate  charts  to  represent  the  data  in  low  dimensions.  It  also  describes 
algorithms  for  performing  function  estimation  for  an  arbitrary  function  defined  on  the  data  cloud. 


Use  or  disclosure  of  data  contained  on  this  sheet  is  subject  to  restrictions  on  the  title  page  of  this  report. 
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4  Methods,  Assumptions  and  Procedures 


4.1  Multiscale  Heat  Kernel  Coordinates 

The  Multiscale  Heat  Kernel  Coordinates  (MHKC)  algorithms  are  based  on  theoretical  results 
presented  in  [1],  The  basic  algorithms  was  described  in  [11].  We  use  the  same  notation  in  this 
report. 

Given  a  set  of  n  data  points  {x1,x2, ...  ,xn}  in  Rd,  we 

1.  Normalize  the  data  by  centering  the  cloud  at  zero  and  scaling  to  fit  the  data  in  a  ball  of 
unit  variance, 

2.  Compute  the  transition  probability  matrix  and  its  SVD 

to  obtain  the  eigenvalues  (AJ  and  associated  eigenvectors  {Vj.}.  The  MHKC  embedding  in  r- 
dimensions  for  the  point  xt  is  then  given  by 

y.  =  (e-*lt.vil,e-X2t.vi2,...,e-Art.vir) 

The  heat  kernel  function  is  defined  as 


/c(x,y)  =  exp 

for  any  two  points  x  and  y.  Here,  £  is  a  constant  (data  dependent)  representing  the  kernel  window 
size. 


4.1.1  Coordinate  Charts 

We  now  describe  the  algorithm  to  create  canonical  coordinate  charts  in  2  dimensions.  This  can 
be  easily  extended  to  r  dimensions.  The  first  step  in  to  obtain  a  pair  of  points  for  each  dimension. 
Next,  we  use  these  pairs  as  reference  points  to  represent  the  entire  data  set.  Specifically, 


1.  Center  the  2-dimensional  data  cloud  (yj  at  the  origin  by  subtracting  the  mean  value 

2.  Find  a  pair  of  points  (w0,  wx)  in  the  dataset  such  that  (^e~Alt.  Vi(w0),  e~Alt.  v2(w0 ))  and 
^e~Alt.v1(w1) ,e-A2t.r’2(w1)^  are  closest  to  the  points  (l,  —  ^  and  respectively 

3.  Repeat  step  2  to  find  another  pair  of  points  (z0,  zx}  corresponding  to  the  points  l) 
and  Q,  l)  respectively 


4. 

5. 


The  coordinate  chart  for  the  point  xt  is  then  given  by 


kjxj.w^ 
k(xi,w0)  ’ 


log 


fc(xi,z1)\ 

k(xi,z0)J 


Center  the  new  coordinates  at  zero. 


Use  or  disclosure  of  data  contained  on  this  sheet  is  subject  to  restrictions  on  the  title  page  of  this  report. 
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Note :  The  MSVD  algorithm  may  be  subsequently  run  on  the  coordinate  chart  to  read  off  the  local 
changes. 

4.1.2  Function  Estimation 

Assume  that  we  are  given  the  function  values  {/(Xj)}  for  an  arbitrary  unknown  function  /.  The 
objective  is  to  compute  the  value  of  /(x)  for  a  new  point  x  that  is  not  in  the  dataset.  The  function 
estimation  algorithm  is  based  on  the  Donoho-Johnstone  technique  for  function  estimation  in  the 
context  of  wavelets. 

The  function  estimate  at  a  new  point  x  is  given  by 

*(  ,  _  E?=ifc(x,Xt)./(Xi) 

nX)  Z?=1fc(x,xt) 

Note-1:  The  above  equation  is  also  a  function  of  the  window  size  £.  To  obtain  a  stable  function 

estimate,  we  require  the  estimate  value  to  approximately  remain  the  same  across  successive 
111 

values  off  =  1, ... 

2  4  8 

Note-2:  The  above  equation  corresponds  to  a  single  time  step  in  the  diffusion  process  with  the 
constraint  that  the  random  walk  does  not  revisit  the  new  point  x.  For  arbitrary  time  t  (keeping 
the  constraint),  we  simply  replace  /c(x,  Xj)  by  the  i-th  component  of  the  vector  given  by 

{k(x,  Xi),  k(x,  x2), ... ,  k(x,  xn )).  Pt_1 


where  P  is  the  Markov  transition  probability  matrix  for  the  original  dataset. 
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4.2  Deliverables  /  Milestones 


Date 

Deliverables  /  Milestones 

Status 

Oct  2010 

Progress  report  for  period  1,  1st  quarter 

Jan  2011 

Progress  report  for  period  1,  2nd  quarter  /  complete  randomized  matrix  decompositions  task 

Apr  201 1 

Progress  report  for  period  1,  3rd  quarter  /  complete  approximate  nearest  neighbors  task 

Jul  201 1 

Progress  report  for  period  1,  4th  quarter  /  complete  experiments  -  part  1 

Oct  201 1 

Progress  report  for  period  2,  1st  quarter 

Jan  2012 

Progress  report  for  period  2,  2nd  quarter  /  complete  multiscale  S  VD  task 

Apr  2012 

Progress  report  for  period  2,  3rd  quarter 

Jul  2012 

Progress  report  for  period  2,  4th  quarter  /  complete  experiments  -  part  2 

Oct  2012 

Progress  report  for  period  3,  1st  quarter 

Jan  2013 

Progress  report  for  period  3,  2nd  quarter  /  complete  multiscale  Heat  Kernel  task 

Apr  2013 

Progress  report  for  period  3,  3rd  quarter 

V7 

Jul  2013 

Final  project  report  +  software  +  documentation  on  CDROM  /  complete  experiments  -  part  3 

Use  or  disclosure  of  data  contained  on  this  sheet  is  subject  to  restrictions  on  the  title  page  of  this  report. 
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5  Results  and  Discussion 


We  described  two  additional  MHKC  algorithms  -  one  to  build  stable  canonical  coordinate  charts 
for  representing  the  dataset,  and  the  other  to  obtain  function  estimates  for  an  unknown  function 
defined  on  the  dataset  We  will  experimentally  evaluate  both  these  techniques  against  real-world 
datasets. 


Use  or  disclosure  of  data  contained  on  this  sheet  is  subject  to  restrictions  on  the  title  page  of  this  report. 
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6  Conclusions 


The  project  is  on  track  with  design/implementation  of  the  new  multiscale  heat  kernel  coordinates 
algorithms.  We  will  finish  up  pending  work,  start  writing  up  the  final  report  and  packaging  all 
deliverables  in  the  upcoming  final  quarter. 

No  problems  are  currently  anticipated. 


Use  or  disclosure  of  data  contained  on  this  sheet  is  subject  to  restrictions  on  the  title  page  of  this  report. 
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